Welcome to NetBots 1.0

your automated internet agent

NetBots is a program that lets you send out little "agents" over the Internet to accomplish tasks for you. This document will tell you how to use NetBots and how to pay for it. Additional information and updates can be found at the NetBots web page: http://www.printerport.com/klephacks/netbots.html

Table of Contents

• Introduction
• Requirements
• Quick Start
• PortBots
• FingerBots
• PingBots
• Sample Usage
• Technical Details
• Morality and Legality Issues
• Common Problems Experienced
• Questions and Answers
• Registering NetBots
• About the Author
• Special Thanks
• Legal Stuff

Since NetBots is fully customizable and works with the Internet at a fairly low level, it can very easily be used incorrectly or even illegally. If you're not familiar with how TCP/IP works, please read this entire document before using NetBots.

1. You should know what finger is
2. You should be somewhat familiar with how the Internet works
3. You should know that different Internet services like FTP, telnet, and the web all run on separate "ports". For example, port 80 is the HTTP port. So when you connect to a site with Netscape, you are really connecting to some computer on port 80.
4. You should NOT use NetBots maliciously or to spy on people. If you're unsure, read the Morality and Legality section.

1. Launch NetBots by double-clicking on the icon
2. Send a new bot by selecting an item from the "Bot Queue" menu
3. Your bot will be added to the queue, which runs every 2 minutes
4. You can run your bot immediately by selecting "Run Queue" from the "Bot Queue" menu
5. You can change the queue delay time by selecting "Preferences" from the "Edit" menu
6. You can get rid of all the existing bots by selecting "Clear Queue" from the "Bot Queue" menu
7. You can halt the queue while it's running by pressing and holding escape (this is meant for emergencies only and can sometimes cause your machine to hang)

PortBots allow you to send any string of text to any TCP/IP port of any Internet host. Each time the bot runs, it will search for a key phrase in the text it receives from the host. If it finds a match, it lets you know. If not, it keeps searching.

When you select "New PortBot" from the "Bot Queue" menu, you are prompted for the following information:

Host: This is the Internet host you're going to connect to. NetBots accepts standard host names as well as dotted-decimal IP addresses.

Connect to Port: This is the TCP/IP port you're going to connect to. Some of the popular services are listed in the popup menu next to the port field. This is an example of where you can cause problems. NetBots allows you to attempt to connect to any port. You can put in ports that don't exist. If you're lucky, you'll get an error message. If you're unlucky, you might crash your machine. It's a good idea to stick to known ports, like the ones listed. The most popular of those are Finger (79) and HTTP (80).

Send String: This is the string of text you're going to send to the host. It must be terminated appropriately! Most services require a UNIX-style carriage return (¥r¥n) after the string. If you leave it off, you won't get anything back from the host and NetBots will terminate the Bot after its first run. In addition to the carriage return characters, you can also send tabs (¥t) if you need to. The only special characters NetBots recognizes are ¥r, ¥n, and ¥t.

Search String: This is what you are looking for in return. This is where you get creative. For example, you could run a PortBot that runs on port 79 (finger) and looks for the string "New mail". On most UNIX hosts, this would let you know when you have new mail. The bot would keep running until it found that string, so you'd know the instant you got a new mail message. More examples of how to use PortBots creatively are found in the Sample Usage section.

Retire after first success: Leave this box checked if you want the Bot to automatically retire after it finds the string (it will remain in the queue until it is found). Be warned that if this box is not checked, the Bot will never die out automatically. It will just let you know every single time it finds your search string.

Alert Text: This is what you'll see when your bot makes a successful run. NetBots uses the Macintosh Notification Manager to alert you of a Bot's status. No matter which application you're in, NetBots will display this message. This allows you to easily run NetBots in the background while you're working.

FingerBots are pretty easy to use. You select "New FingerBot" from the "Bot Queue" menu, then enter the person's email address in the fields. The Alert Text field allows you to customize how you will be notified when the user logs on. When you click "Send", the bot is added to the queue.

NetBots pauses two minutes between queue runs by default. You can customize this delay by selecting "Preferences" from the "Edit" menu. The first run will be immediately after the first bot is added to the queue. When a FingerBot has a successful run (i.e. the user has logged on), it self-destructs (by removing itself from the queue and not running anymore) and notifies you.

It's worth noting that FingerBots are not fool-proof. You can easily trick a FingerBot by putting the phrase "On since" somewhere in your .plan file (the file that UNIX displays when you finger a user). However, it's very unlikely that you would encounter such a circumstance.

PingBots aren't really a true implementation of what's commonly referred to as "ping". Instead, they just try to connect to a certain port of a host. NetBots uses port 79 (finger) by default, but you can change it to your taste. Any open port will do. Port 79 is usually open on most UNIX hosts. In contrast, most Macs have no open ports, so PingBots won't work. If the Mac in question is a web server, use port 80. If it's an FTP server, use port 21.

To send a PingBot, select "New PingBot" from the "Bot Queue" menu. Enter the hostname (regular or dotted-decimal) and any open port. If you choose "Check for down", the PingBot will let you know when the host goes down. If you select "Check for up", the PingBot will let you know when the host comes back online. Either way, the bot will alert you with the "Alert Text" when it has a successful run and will then retire automatically.

The PingBot will run, along with the rest of the bots in your queue, every two minutes by default. You can change this by selecting "Preferences" from the "Edit" menu. PingBots are set to determine if a host is up or down by trying to connect for up to about 30 seconds. If that amount of time goes by and no connection has been established, the PortBot assumes the host is down.

Check for web page updates - If you run a PortBot on port 80, you can repeatedly scan web pages for a string of text. A lot of pages are updated daily. If you need to know immediately when a page is updated, just enter the date (in whatever form that page uses) in the search string and the bot will alert you when the page is updated. It's really easy to run PortBots on port 80. To get a web page, just use the "GET" command. For example, "GET /index.html¥r¥n" is the send string to get the index.html file from a server. Remember to end the send string of any PortBot with "¥r¥n".

Single-run bots - While the idea of NetBots is to run them repeatedly until something changes, it can be useful to run them a single time. For example, if you wanted to know if your name was on a particular web page, but the page was long and you didn't want to run Netscape, just run a PortBot, as described above, and see if it alerts you the first time through.

A better BIFF - biff is the UNIX program that lets you know when new mail arrives. If your telnet window is in the background, you might not notice however. Instead, you can run a PortBot on port 79 (finger) that looks for the string "New mail". Most UNIX hosts put "New mail received ...." in your finger information when you have new mail.

A talk assistant - This is why I originally started NetBots. If you want to talk to somebody over the Internet using a program like ytalk for UNIX or talk for the Mac, you want to know when they log in. Instead of fingering them every few minutes, I just run a FingerBot and I'm notified when they log on.

Web traffic monitor - If you run a web server but you're unsure if you can handle all the accesses you're getting, try running a PingBot on port 80. Select "check for down" and you'll be notified if NetBots was unable to establish a connection to your web server.

Web site ping - Here's a good one. If you want to get on a web site, but it's busy, just run a PingBot on port 80 with the URL as the alert text. Go on with your work/surfing and when the site is available, you'll get a message with the URL and you can go there. It works great, especially because Netscape would hang in the same situation.

I should also note that for the TCP/IP code, I used Simple Sockets, the excellent sockets library by Michael Trent. Simple Sockets allowed me to use some of the code that I had already written in UNIX. Michael was also very helpful when I was just getting started with NetBots. After struggling with Open Transport and MacTCP for several days, Simple Sockets made it possible to actually establish a connection.

Here's some other miscellaneous technical information: NetBots uses MacTCP, not Open Transport. I will port it over to OT in the future or when Simple Sockets gets ported. The default memory partition should be fine for the shareware version since it places a limitation on the number of bots (the registered version allows an infinite number of bots as long as there's enough memory). The bot queue runs in no particular order -- you shouldn't depend on your bots running in the order you created them. The RegisterBot pops up occassionally to remind you that you're using shareware. To disable it, see the Registering NetBots section. NetBots went through a beta testing period with some of the best beta testers I've ever worked with. Much of the interface was designed by Darien Kruss, and some of the graphics were done by Dave Laster.

First off, most Internet admins won't notice if you're running NetBots on their site. Still, you should ask first if you're going to be running it for an extended period of time or in a way that seems questionable. Running a FingerBot every 20 minutes for a few hours probably isn't a problem. However, some admins are picky about any sort of automated client running on their machine. Some are even known to send threatening messages to users who run programs like NetBots to continually finger a user. This is precisely why NetBots doesn't allow you to run the queue any more often than every two minutes. I don't think anybody should have a problem with this, but again, you are responsible for your own actions. If you are asked to stop, then stop. If you're unsure about a bot you're about to send, ask the admin first.

Most of the harm that NetBots can do is only in using up CPU time by repeatedly sending requests on ports. On most modern computers, this isn't a problem and NetBots won't use up many resources. There's nothing you can do with NetBots that you couldn't do manually, so obey the same laws. Just because a bot is doing something, doesn't mean it's not illegal and certainly doesn't mean it can't be traced back to you. Use NetBots to help manage your sites and to locate users. Don't use it to spy on people or attempt to break into systems. It will get you in trouble and it's not even all that great for those purposes. Remember: respect the wishes of Internet admins and be very careful with NetBots.

Make sure that you have established a TCP/IP connection. NetBots will attempt to establish one if you're not connected, but if your connection is up but unreliable, NetBots may hang, pause, or quit.

I search for a string using a PortBot and it's not found even though I know it's there

There are a few ways that a PortBot can be tricked. For one thing, PortBots are case-sensitive. The case must match for the string to be found. The other way is more subtle. PortBots receive information back from the host in blocks of up to 5 kilobytes. So if the text you're receiving (a web page for example) is 7 kilobytes, you'll get two blocks -- one of 5k and one of 2k. If the text you're searching for starts at the end of the first block and ends at the beginning of the second block, NetBots will miss it entirely. Future versions will fix this problem. Please note, however, that 5k is pretty large, and should cover most types of transactions. Finally, there's one other possibility. The string you send to a PortBot must be terminated appropriately! Almost always, this will be by ending it with ¥r¥n, but since NetBots lets you do whatever you want, it won't add this string by default.

FingerBots don't seem to work on some users

FingerBots are really PortBots that look for the string "On since" when fingering a user. Most UNIX-like hosts return "On since" when a user is online. Many schools still use VMS instead of UNIX, which does not do this. Even some versions of UNIX don't comply with this method. For those, you'll have to use a PortBot accordingly. In some cases, even a PortBot won't work. For example, a human can easily determine if a VMS user is online by looking at the result of a finger. But a Bot wouldn't know what to look for. The best guess would be "*DCL*", which is likely to appear when the user first logs on. For example, here's the result of fingering my RIT account when I first log on:

[rit.edu]
SJKED    Kleper, Sj          27C12DF6 *DCL*         .nty26 TCP: 
ppp4.cs.rit.edu

[No plan]

When I halt the bot queue by holding down escape, my Mac freezes

This may happen from time to time if you halt the queue at a critical time in the connection process. I've made every attempt to eliminate this bug and it won't show up in most circumstances. Think of the escape key as an emergency halt that will almost always work, but shouldn't be relied upon. For those interested in why it hangs, the reason is that I didn't write most of the low-level network code for NetBots. To implement features like time-outs (for PingBots) and escapes (to cancel the queue), I had to hack around in some code I didn't write. The sockets library I used wasn't designed to be interrupted while connecting/sending/reading so it can get tripped up.

A: My name is Scott J. Kleper. I'm a sophomore at Stanford University, originally from Pittsford, New York. I'm 19 years old and I've been programming Macs for about five years. I've been writing shareware for almost as long. My other programs include MacFolklore, HTML Markdown, HTML Markup, Jot, and PowerSpaceTabsPlus. All are available from my home page at http://htc.rit.edu/scott.html

Q: Why did you write NetBots?

A: During the 95-96 school year, I was fortunate enough to have an internship at Be, Inc. in Menlo Park, CA. One of my projects was to write a Finger client for the BeBox. Since BeOS is POSIX-compliant, I developed the program concurrently for UNIX and BeOS. When I was done, I modified the UNIX version to analyze the data that was returned so I could be notified when a user logged on. As I started to think of similar programs that I'd like to write, I decided to unify them as NetBots.

Q: Are there any similar programs?

A: Not really (at least not that I've seen). I have a theory why -- in the past, most people who were on the Internet were UNIX folks. To them, NetBots would seem pretty trivial. They could write UNIX shell scripts that would do most of the same tasks (without the cool interface). However, with the growth or SLIP/PPP accounts and the recent explosion of the Internet, UNIX and Internet are no longer synonymous. People are using Macs on the Net and NetBots is the most elegant solution I've seen for repeated connections.

Q: How much trouble can I get in for using this?

A: None if you use it correctly. NetBots doesn't do anything that you couldn't do yourself from a UNIX prompt. NetBots just automates it and makes it repetitive. Some admins get nervous about any sort of automated "robot" connecting to their site repeatedly. But since you can't connect more often than once every 2 minutes, you should be fine. As a disclaimer, you are responsible for how you use NetBots.

Q: Why aren't there other kinds of Bots?

A: NetBots was developed as a fully object-oriented program. To the user, this probably means nothing. But to a programmer, this means that I can easily add new Bots to the existing front end. It also means that I can easily modify the interface and keep the program the same. These are good things. I won't commit to developing future versions, but I will say that it's very likely. Some of the features I'd like to include in the future are:

• Multithreading so you can run Bots concurrently and queue up new ones while the queue is running
• Native Open Transport code
• InfoBots, which would go out and gather as much information as possible about a host or user. These would be one-run bots.
• Various WebBots, which would actively find dead links in your site, active searching, etc.
• Modify the PingBot to implement the true Ping protocol (UDP packets)
• A BotManager that would allow you to view all your bots, disable bots, modify bots, etc.
• More statistics, and a log file
• Give bots names/numbers to identify them
• Some sort of plugin architecture to let programmers write their own bots
• MailBot which would check a POP account for new mail and give you the subject/from line
• TouchBots, which would check FTP sites and Web pages for modifications by comparing them to the last seen copy

Q: How can I halt the bot queue?

A: While the queue is running, you can press and hold escape to attempt to halt the queue. This will usually work. Please note that if you have several bots in the queue, you might need to hold down the escape key for a while. Your Mac will beep as each consecutive bot in the queue is skipped.

Q: Where's support for AppleScript and other new Apple technologies?

A: AppleScript would be a good thing for NetBots, but I didn't have time to include scripting support in this version. It is planned for a later release. As far as other technologies are concerned, if there's something you want to see, send me the idea and I'll see what I can do. Send email to klephacks@kagi.com

Q: I work for a user group, CD-ROM publisher, or magazine. Can we give out your program?

A: If you're going to be distributing it electronically or on disk, you may distribute NetBots 1.0 without notifying me. However, I still request that you send me a quick note letting me know where it's going to be distributed. If you are going to write an article or review of NetBots, I really really really want to read it. Please please please send me a copy of the article. If you're going to be distributing NetBots 1.0 on a CD-ROM, you must notify me first. I will generally grant permission for distribution, but I want to know which CD it's going to be on. Send permission requests, reviews, etc. to:

Scott J. Kleper
Attn: NetBots 1.0
klephacks@kagi.com
134 Caversham Woods
Pittsford, NY 14534-2834
USA

Q: I want to learn more about TCP/IP programming in general. Where can I look?

A: I'm probably not the best source for TCP/IP programming questions because I still use UNIX-style sockets, not MacTCP or Open Transport calls. The best resource is the Macintosh TCP/IP Programmer's Guide.

Q: Why does this "Register Bot" come up randomly? Just to annoy me into giving you money?

A: The RegisterBot is not intended to annoy you. And actually, it doesn't come up randomly. It shows itself the first time you run through the queue, and every seventh time thereafter. It will show up less often if you have a longer wait between queue runs. The reason I have the RegisterBot is just to remind you that the program you're using is shareware. Writing software is one of my favorite hobbies and I love getting feedback from users who have used my programs to save time. But software development is time-consuming and expensive, especially for a kid trying to work his way through college. All I ask is that if you use NetBots more than a couple of times, you register as described in the Registering NetBots section.

Q: Can I have the source code so I can write my own version of NetBots?

A: No. NetBots 1.0 is a copyrighted product and I will not release the source code. However, I am sympathetic towards other people working on similar projects and I'd be happy to discuss techniques or provide help with your program. Send me email. If you're interested in writing a similar program from scratch and you're familiar with the UNIX socket interface, you might want to try Simple Sockets, the socket package I used.

If you do decide to register NetBots, you will receive the registered version, which will allow you to add as many bots to the queue as memory permits. It will also disable the RegisterBot. If you provide your email address with your registration, I'll even notify you of maintenance updates.

Registration costs $15 for a single-user license. This means that one person can use NetBots on one Macintosh. If you're in a multi-user environment, or want to have NetBots installed on multiple machines, you may purchase multiple licenses at $15 each or a site license for $200.

The preferred way of registering is to send me a check directly. Make your $15 check payable to "Scott J. Kleper" and please include your email address if possible. I can also take traveler's cheques, money orders, and international money orders in US dollars.
Send checks to:

Scott J. Kleper
134 Caversham Woods
Pittsford, NY 14534-2834
USA

The alternate way of paying is through the Kagi shareware payment service. Paying through Kagi allows you to use a check, credit card, or electronic payment. To register through Kagi, just double-click the "Register KlepHacks" program that came with NetBots. If you didn't get the Register program when you downloaded it, it is available from:

ftp://htc.rit.edu/pub/register-klephacks.hqx

You can also register on the web through Kagi, which is something new. You have to pay by credit card if you use this method, but it's quick and easy. Just go to the KlepHacks Register Page for more information.

The register program is pretty easy to use, but if you need help, please read the "KlepHacks Register Help" file. Again, please include your email address so that I can send the registered version to you as quickly as possible.

One last thing about registering. I really appreciate it when people send comments about and suggestions for the program. I read them all and if you supply an email address, I'll probably contact you. If you use the Kagi method, you can still send comments.

If you have any questions, feel free to send me email at klep@cs.stanford.edu

This program was tested by a small number of dedicated beta users, most of whom were instrumental in the testing of HTML Markup 2.0, my last shareware program. Darien Kruss took it upon himself to send me a complete interface for NetBots, and I implemented as much of it as I could. Early versions of NetBots were all grey windows with black Chicago text. Thanks to Darien, the dialogs are much more elegant. Some of the graphics were done by Dave Laster. Just about everybody contributed something to the program you now have installed. Here are the names of the beta testers:

• Darien Kruss
• Matt Garlinghouse
• Dave Laster
• Frank Gerratana
• William Wilson

I'd also like to thank my mom. Thanks, mom!

No warranty is included with this program. Use it at your own risk. There are no known bugs with this program. However, the author is not responsible for any problems caused by it. The user assumes all responsibility for abuse/misuse of this program. You are responsible for your own actions and the author is not responsible for any unethical or criminal actions performed with the aid of NetBots.

This program may be included in online file areas and archives. It may be distributed through user groups and shared with other users. CD-ROM publishers MUST contact me at klephacks@kagi.com before including NetBots 1.0 on a CD-ROM product.

If you would like to review NetBots for an online or traditional magazine, please contact me so that I can see a copy. □□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□p□□□p□□Z□□…ｰ□resume.html□□□TEXTR*ch□…… docs.html□□□TEXTMOSS□…………□□□□□TEXTMOSS□…………□□□□□□□□□\□□□□□□ｭ□Zﾀ□□⑨□□□ﾊ□□□□□□□□□□□□…ｰ□sjk.htmlk□□□TEXTR*ch□…………□□□□ⅳ□□□□□□□"□□□□□□□□"ｫ‡]?ｬä□｡□□□□□□□□□□□□□□□□□□□□□% □□□□□□□□□% □□□□□□□□□□□□□□□□□□□□□□□□□□□□H⇥Monacoﾙ!□□ﾘﾛâ□ﾘ錘□ﾘô\□□□□□□□□□□□□□□C□□□ﾌ□□C□□□ﾌ□□ｮ□B*□o0□o0□□□G□□□□□□R*ch\□□□□HH□□□□□ﾚ□(…á…á□ù□E□G□(□ü□□□□HH□□□□□ﾘ□(□□□□d□□□□□□□□□□□□'□□□□□□□□□□□□□□□□`□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□Monaco˜□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□⇥□□□□ Helvetica□M□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□Confidential□ H□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□\□□\□□\□□\□□□□□□□□□□□□□□□□□D□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□p□□□p□□Z□ﾘù8-è□□□□Z□□MWBB□□□□MPSR□□&BBST□□2□ð……□□□□□□□□□í……□□□□□□□□\……□□T□ﾘô\□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□